When a sequence of image frames depicting a scene is captured, differences between consecutive frames are often introduced, due to movements between the imaging device used to capture the image frame sequence and the scene that is being imaged. Such differences may also be referred to as image jitter and is perceived when displayed as a small image motion. The scene may for example have changed from one frame to the next due to changes in light in the imaged scene or movements of depicted objects. Also, in most cases the imaging sensor will not have been kept perfectly still. If the imaging device is handheld, it is evident that there will be movements caused by the user of the imaging device. If the camera is stationary, for example on a stand, vibrations of the imaging device or the surroundings may cause movements of the imaging sensor.
In order to compensate for such relative movements between the imaging device and the imaged scene, image stabilization methods are commonly used. Image stabilization in the context of this description may also be referred to as motion compensation.
There are many different ways of stabilizing images known in the art. Most of them relate to stabilization of visual light images. However, a good image stabilization algorithm for visual images will not necessarily be a good image stabilization algorithm for infrared (IR) images, because of the different properties of visual light images and IR images.
For instance, visual light images typically have a smaller dynamic range than IR images, since visual images are usually represented using 8 bits of information per pixel (255 colors), while the information in infrared images pixels is usually represented by 16 bits of information, corresponding to approximately 65000 detectable levels of IR radiation. This refers to the underlying information comprised in the IR image representation. Of course, when an IR image is displayed on a display device it will be first have been converted to a visual representation where every pixel comprises 8 bits of information, thereby being enabled to display 256 color levels. For stabilization purposes however, it is desirable to stabilize on the underlying IR data and not on the compressed visual interpretation of the data. This means that image stabilization methods used for visual images will in many cases become much too computationally expensive when the same methods typically used for 8 bit images are applied on 16 bit images.
How an IR image is visually represented and displayed depends on the underlying IR image data, but adjustments may be made to the visual representation before it is presented to a user. For instance, the number of colors, or representation levels, is typically reduced to 255. However, as described above it is the underlying image data that is used for IR image stabilization.
Also, visual light image data is more alike from frame to frame in an image frame sequence, since the shutter of the visual light imaging device is adapted based on the incoming light, compensating for changes in lighting, so that there is a smooth transition from frame to frame as the visual light image frame sequence is displayed to a user. This likeness between image frames is for instance apparent by regarding the histograms for subsequent visual light image frames, wherein the histograms typically correspond well to each other. For IR images on the other hand, the underlying IR image data is based on the entire range of detected IR radiation, corresponding to the temperatures of the objects in the images scene. Therefore, if the temperatures of objects in the scene change rapidly, the IR image data will change accordingly. IR images typically also comprise more noise than visual light images and have a lower signal-to-noise (SNR) ratio, which may complicate image stabilization based on matching between subsequent image frames.
In order to stabilize images with respect to each other, it is common to identify corresponding features or objects in the images and use the position of corresponding features or objects to shift one image with respect to the other in order to achieve proper registration.
In order to reduce the computational effort required for such feature or object recognition and/or identification, it is known in the art to apply a Radon transformation to the images, in their original form, before they are compared and stabilized. Thereby the information in each m×n image is compressed into two vectors of the sizes m×1 and n×1, respectively.
A feature or object recognition method will typically recognize features or objects with large contrast compared to surrounding pixels, rather than features or objects with low contrast compared to the surroundings. For IR images, this means that features or objects with high temperature, or high intensity, are more likely to be recognized than other features or objects in the image. In order words, hot objects are weighted higher than cold objects in the recognition process. In order to reduce this weighting of hot objects, derivatives or gradient-based methods may be used. However, such methods are quite complex and therefore computationally expensive to use for image preprocessing.
Another possible approach towards reducing the weighting of hot objects is to use mean value or median value calculations of the images, and normalize the images based on the calculated values. In order to calculate the mean or median value of an image, the entire image must be stored in a memory and loaded in its entirety, in other words all pixel values must be accessible. For the case of median value calculation all values must further be sorted. Therefore, preprocessing the images by calculation of mean or median values will increase the computational load significantly.
Therefore, there is a need for improved image stabilization that is adapted to the properties of IR images.